{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8aea7cf6-81de-48af-8470-f5b9bf229113",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Installing Feast\n",
    "Feast is a python dependency so we have to install it using `pip`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ceb60bdb-0c90-4991-91df-dc7334ff3a93",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.2.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "feast package is installed\n"
     ]
    }
   ],
   "source": [
    "# WE MUST ENSURE PYTHON CONSISTENCY BETWEEN NOTEBOOK AND FEAST SERVERS\n",
    "# LAUNCH THIS NOTEBOOK FROM A CLEAN PYTHON ENVIRONMENT >3.9\n",
    "%pip install -q feast==0.40.1\n",
    "# grpcio is needed as a dependency in the later section of the example to run the feast registry server.\n",
    "%pip install -q grpcio\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b70c61f-1e94-4354-8dcb-3a57be140d13",
   "metadata": {},
   "source": [
    "## Creating and initializing Feast project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0fcecd34-ff0b-4528-88a5-3125187241f6",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/opt/app-root/src/feast/examples/rhoai-quickstart'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Displaying the current directory. We will know where the feast files will be created so that we can review them using jupyter console or explorer\n",
    "%pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1ad595e5-d98a-4f9c-bead-f2d0ad274855",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Creating a new Feast repository in \u001b[1m\u001b[32m/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project\u001b[0m.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Creating the feast repository. If there is already existing repository then removing it first.\n",
    "!rm -rf my_feast_project\n",
    "!feast init my_feast_project"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3beb2f85-3e28-4afa-8227-30271dd00c7f",
   "metadata": {},
   "source": [
    "Above output displays where the feast repo has been created. It may differ based on the environment configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "308610d4-9174-4972-a785-237c0b0477d8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/app-root/lib64/python3.9/site-packages/IPython/core/magics/osm.py:417: UserWarning: using dhist requires you to install the `pickleshare` library.\n",
      "  self.shell.db['dhist'] = compress_dhist(dhist)[-100:]\n"
     ]
    }
   ],
   "source": [
    "# Going to change the current directory to feature_repo so that we can execute feast CLI commands.\n",
    "%cd my_feast_project/feature_repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7f5212d8-cd46-4017-a762-91a4fb56e0d5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\n",
      " +-- data\n",
      " +--  |-- driver_stats.parquet\n",
      " +-- __init__.py\n",
      " +-- __pycache__\n",
      " +--  |-- __init__.cpython-39.pyc\n",
      " +--  |-- test_workflow.cpython-39.pyc\n",
      " +--  |-- example_repo.cpython-39.pyc\n",
      " +-- feature_store.yaml\n",
      " +-- example_repo.py\n",
      " +-- test_workflow.py\n"
     ]
    }
   ],
   "source": [
    "# Inspect the feast repo path files. Displaying folder strucuture as tree. Going to describe each file/folder purpose.\n",
    "!find . | sed -e 's/[^-][^\\/]*\\// |-- /g' -e 's/|-- \\(.*\\)/+-- \\1/'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa8d4c3-8f7d-4758-8048-945ebd0af53f",
   "metadata": {
    "tags": []
   },
   "source": [
    "Now the feast repo has been created for you. Running the `feast init` command populated the directory with an example feature store structure, complete with example data.\n",
    "\n",
    "We are defining an entity for the driver in the current example. You can think of an entity as a primary key used to fetch features. Rest of the example will work on the driver data. All the data is coming from the `data/driver_stats.parquet` file which will act as offline store in our example.\n",
    "\n",
    "Inspect the below files before going further in the current example.\n",
    "\n",
    "`data` contains the parquet file data used to demonstrate this example.\n",
    "\n",
    "`example_repo.py` file will have the code to create feast objects such as FeatureView, FeatureServices and OnDemandFeatureViews required to demonstrate this example.\n",
    "[my_feast_project/feature_repo/example_repo.py](./my_feast_project/feature_repo/example_repo.py)\n",
    "\n",
    "`feature_store.yaml` file will have all the configurations related to feast.\n",
    "[my_feast_project/feature_repo/feature_store.yaml](./my_feast_project/feature_repo/feature_store.yaml)\n",
    "\n",
    "`test_workflow.py` contains the python code to demonstrate run all key Feast commands, including defining, retrieving, and pushing features.\n",
    "[my_feast_project/feature_repo/test_workflow.py](./my_feast_project/feature_repo/test_workflow.py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f0a63ea0-93fa-4d4e-b12d-6caef86478f9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "project: my_feast_project\n",
       "# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)\n",
       "registry: data/registry.db\n",
       "# The provider primarily specifies default offline / online stores & storing the registry in a given cloud\n",
       "provider: local\n",
       "online_store:\n",
       "    type: sqlite\n",
       "    path: data/online_store.db\n",
       "entity_key_serialization_version: 2\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "!cat feature_store.yaml"
   ]
  },
  {
   "cell_type": "raw",
   "id": "7094b08a-24bb-4608-9182-b8d894f85e51",
   "metadata": {},
   "source": [
    "The default feature store configuration uses the sqllite as registry and online store, dask as offline store. \n",
    "You can find more information: https://docs.feast.dev/reference/providers/local."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd23ce19-03fa-4353-86d3-261c7f514fd8",
   "metadata": {},
   "source": [
    "File `data/driver_stats.parquet` is generated by the `feast init` command and it acts a historical information source to this example. We have defined this source in the [my_feast_project/feature_repo/example_repo.py](./my_feast_project/feature_repo/example_repo.py) file.\n",
    "\n",
    "```python\n",
    "driver_stats_source = FileSource(\n",
    "    name=\"driver_hourly_stats_source\",\n",
    "    path=\"/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo/data/driver_stats.parquet\",\n",
    "    timestamp_field=\"event_timestamp\",\n",
    "    created_timestamp_column=\"created\",\n",
    ")\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6dc4f196-d9ec-421f-9cce-16678b783ea7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_timestamp</th>\n",
       "      <th>driver_id</th>\n",
       "      <th>conv_rate</th>\n",
       "      <th>acc_rate</th>\n",
       "      <th>avg_daily_trips</th>\n",
       "      <th>created</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2024-09-09 17:00:00+00:00</td>\n",
       "      <td>1005</td>\n",
       "      <td>0.373758</td>\n",
       "      <td>0.475401</td>\n",
       "      <td>354</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2024-09-09 18:00:00+00:00</td>\n",
       "      <td>1005</td>\n",
       "      <td>0.057971</td>\n",
       "      <td>0.375569</td>\n",
       "      <td>517</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2024-09-09 19:00:00+00:00</td>\n",
       "      <td>1005</td>\n",
       "      <td>0.383832</td>\n",
       "      <td>0.323274</td>\n",
       "      <td>484</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2024-09-09 20:00:00+00:00</td>\n",
       "      <td>1005</td>\n",
       "      <td>0.403390</td>\n",
       "      <td>0.570664</td>\n",
       "      <td>634</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2024-09-09 21:00:00+00:00</td>\n",
       "      <td>1005</td>\n",
       "      <td>0.536741</td>\n",
       "      <td>0.645107</td>\n",
       "      <td>128</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1802</th>\n",
       "      <td>2024-09-24 15:00:00+00:00</td>\n",
       "      <td>1001</td>\n",
       "      <td>0.534048</td>\n",
       "      <td>0.621612</td>\n",
       "      <td>511</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1803</th>\n",
       "      <td>2024-09-24 16:00:00+00:00</td>\n",
       "      <td>1001</td>\n",
       "      <td>0.776248</td>\n",
       "      <td>0.120384</td>\n",
       "      <td>311</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1804</th>\n",
       "      <td>2021-04-12 07:00:00+00:00</td>\n",
       "      <td>1001</td>\n",
       "      <td>0.058821</td>\n",
       "      <td>0.109781</td>\n",
       "      <td>581</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1805</th>\n",
       "      <td>2024-09-17 05:00:00+00:00</td>\n",
       "      <td>1003</td>\n",
       "      <td>0.297863</td>\n",
       "      <td>0.940503</td>\n",
       "      <td>13</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1806</th>\n",
       "      <td>2024-09-17 05:00:00+00:00</td>\n",
       "      <td>1003</td>\n",
       "      <td>0.297863</td>\n",
       "      <td>0.940503</td>\n",
       "      <td>13</td>\n",
       "      <td>2024-09-24 17:07:41.972</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1807 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               event_timestamp  driver_id  conv_rate  acc_rate  \\\n",
       "0    2024-09-09 17:00:00+00:00       1005   0.373758  0.475401   \n",
       "1    2024-09-09 18:00:00+00:00       1005   0.057971  0.375569   \n",
       "2    2024-09-09 19:00:00+00:00       1005   0.383832  0.323274   \n",
       "3    2024-09-09 20:00:00+00:00       1005   0.403390  0.570664   \n",
       "4    2024-09-09 21:00:00+00:00       1005   0.536741  0.645107   \n",
       "...                        ...        ...        ...       ...   \n",
       "1802 2024-09-24 15:00:00+00:00       1001   0.534048  0.621612   \n",
       "1803 2024-09-24 16:00:00+00:00       1001   0.776248  0.120384   \n",
       "1804 2021-04-12 07:00:00+00:00       1001   0.058821  0.109781   \n",
       "1805 2024-09-17 05:00:00+00:00       1003   0.297863  0.940503   \n",
       "1806 2024-09-17 05:00:00+00:00       1003   0.297863  0.940503   \n",
       "\n",
       "      avg_daily_trips                 created  \n",
       "0                 354 2024-09-24 17:07:41.972  \n",
       "1                 517 2024-09-24 17:07:41.972  \n",
       "2                 484 2024-09-24 17:07:41.972  \n",
       "3                 634 2024-09-24 17:07:41.972  \n",
       "4                 128 2024-09-24 17:07:41.972  \n",
       "...               ...                     ...  \n",
       "1802              511 2024-09-24 17:07:41.972  \n",
       "1803              311 2024-09-24 17:07:41.972  \n",
       "1804              581 2024-09-24 17:07:41.972  \n",
       "1805               13 2024-09-24 17:07:41.972  \n",
       "1806               13 2024-09-24 17:07:41.972  \n",
       "\n",
       "[1807 rows x 6 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.read_parquet(\"data/driver_stats.parquet\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92557e68-f044-444e-a68a-31385b95092e",
   "metadata": {},
   "source": [
    "You have not created any feast objects to do that you have to execute command `feast apply` on the directory where `feature_store.yaml` exists. Lets go and do that now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "8e0a05ec-f449-4371-9d55-f30176b51b38",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Below folder is creating interference with the feast apply command so deleting it in case if it exists.\n",
    "!rm -rf .ipynb_checkpoints/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "5c6baa83-9834-46ac-82a4-cc0a9ae33080",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/lib64/python3.9/site-packages/feast/feature_store.py:590: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval\n",
      "  warnings.warn(\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "Created entity \u001b[1m\u001b[32mdriver\u001b[0m\n",
      "Created feature view \u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m\n",
      "Created feature view \u001b[1m\u001b[32mdriver_hourly_stats_fresh\u001b[0m\n",
      "Created on demand feature view \u001b[1m\u001b[32mtransformed_conv_rate\u001b[0m\n",
      "Created on demand feature view \u001b[1m\u001b[32mtransformed_conv_rate_fresh\u001b[0m\n",
      "Created feature service \u001b[1m\u001b[32mdriver_activity_v2\u001b[0m\n",
      "Created feature service \u001b[1m\u001b[32mdriver_activity_v1\u001b[0m\n",
      "Created feature service \u001b[1m\u001b[32mdriver_activity_v3\u001b[0m\n",
      "\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "09/24/2024 06:01:41 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "Created sqlite table \u001b[1m\u001b[32mmy_feast_project_driver_hourly_stats_fresh\u001b[0m\n",
      "Created sqlite table \u001b[1m\u001b[32mmy_feast_project_driver_hourly_stats\u001b[0m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# this command will actual creates the feast objects mentioned in `example_repo.py`\n",
    "!feast apply"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ab04029-b33c-49d4-b107-59ed4c662fa5",
   "metadata": {},
   "source": [
    "## Generating the training Data\n",
    "\n",
    "To train a model, we need features and labels. Often, this label data is stored separately (e.g. you have one table storing user survey results and another set of tables with feature values). Feast can help generate the features that map to these labels.\n",
    "\n",
    "\n",
    "Feast needs a list of entities (e.g. driver ids) and timestamps. Feast will join relevant tables to create the relevant feature vectors. There are two ways to generate this list:\n",
    "\n",
    "* The user can query that table of labels with timestamps and pass that into Feast as an entity dataframe for training data generation.\n",
    "\n",
    "* The user can also query that table with a SQL query which pulls entities. See the [documentation](https://docs.feast.dev/getting-started/concepts/feature-retrieval) on feature retrieval for details\n",
    "\n",
    "Note: we include timestamps because we want the features for the same driver at various timestamps to be used in a model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "7dd75fe4-4eed-4a2a-8714-e8079c4d5772",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----- Feature schema -----\n",
      "\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3 entries, 0 to 2\n",
      "Data columns (total 10 columns):\n",
      " #   Column                              Non-Null Count  Dtype              \n",
      "---  ------                              --------------  -----              \n",
      " 0   driver_id                           3 non-null      int64              \n",
      " 1   event_timestamp                     3 non-null      datetime64[ns, UTC]\n",
      " 2   label_driver_reported_satisfaction  3 non-null      int64              \n",
      " 3   val_to_add                          3 non-null      int64              \n",
      " 4   val_to_add_2                        3 non-null      int64              \n",
      " 5   conv_rate                           3 non-null      float32            \n",
      " 6   acc_rate                            3 non-null      float32            \n",
      " 7   avg_daily_trips                     3 non-null      int32              \n",
      " 8   conv_rate_plus_val1                 3 non-null      float64            \n",
      " 9   conv_rate_plus_val2                 3 non-null      float64            \n",
      "dtypes: datetime64[ns, UTC](1), float32(2), float64(2), int32(1), int64(4)\n",
      "memory usage: 332.0 bytes\n",
      "None\n",
      "\n",
      "----- Example features -----\n",
      "\n",
      "   driver_id           event_timestamp  label_driver_reported_satisfaction  \\\n",
      "0       1001 2021-04-12 10:59:42+00:00                                   1   \n",
      "1       1002 2021-04-12 08:12:10+00:00                                   5   \n",
      "2       1003 2021-04-12 16:40:26+00:00                                   3   \n",
      "\n",
      "   val_to_add  val_to_add_2  conv_rate  acc_rate  avg_daily_trips  \\\n",
      "0           1            10   0.058821  0.109781              581   \n",
      "1           2            20   0.806576  0.493560              809   \n",
      "2           3            30   0.122915  0.132378              636   \n",
      "\n",
      "   conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0             1.058821            10.058821  \n",
      "1             2.806576            20.806576  \n",
      "2             3.122915            30.122915  \n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "import pandas as pd\n",
    "\n",
    "from feast import FeatureStore\n",
    "\n",
    "# Note: see https://docs.feast.dev/getting-started/concepts/feature-retrieval for \n",
    "# more details on how to retrieve for all entities in the offline store instead\n",
    "entity_df = pd.DataFrame.from_dict(\n",
    "    {\n",
    "        # entity's join key -> entity values\n",
    "        \"driver_id\": [1001, 1002, 1003],\n",
    "        # \"event_timestamp\" (reserved key) -> timestamps\n",
    "        \"event_timestamp\": [\n",
    "            datetime(2021, 4, 12, 10, 59, 42),\n",
    "            datetime(2021, 4, 12, 8, 12, 10),\n",
    "            datetime(2021, 4, 12, 16, 40, 26),\n",
    "        ],\n",
    "        # (optional) label name -> label values. Feast does not process these\n",
    "        \"label_driver_reported_satisfaction\": [1, 5, 3],\n",
    "        # values we're using for an on-demand transformation\n",
    "        \"val_to_add\": [1, 2, 3],\n",
    "        \"val_to_add_2\": [10, 20, 30],\n",
    "    }\n",
    ")\n",
    "\n",
    "store = FeatureStore(repo_path=\".\")\n",
    "\n",
    "training_df = store.get_historical_features(\n",
    "    entity_df=entity_df,\n",
    "    features=[\n",
    "        \"driver_hourly_stats:conv_rate\",\n",
    "        \"driver_hourly_stats:acc_rate\",\n",
    "        \"driver_hourly_stats:avg_daily_trips\",\n",
    "        \"transformed_conv_rate:conv_rate_plus_val1\",\n",
    "        \"transformed_conv_rate:conv_rate_plus_val2\",\n",
    "    ],\n",
    ").to_df()\n",
    "\n",
    "print(\"----- Feature schema -----\\n\")\n",
    "print(training_df.info())\n",
    "\n",
    "print()\n",
    "print(\"----- Example features -----\\n\")\n",
    "print(training_df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70322893-947f-46cf-a4a8-6a49d8c799b5",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Run offline inference (batch scoring)\n",
    "To power a batch model, we primarily need to pull features with the get_historical_features call, but using the current timestamp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c1fccc14-9d94-4d1a-a91a-d8b82f8d8044",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----- Example features -----\n",
      "\n",
      "   driver_id                  event_timestamp  \\\n",
      "0       1002 2024-09-24 18:01:58.027897+00:00   \n",
      "1       1001 2024-09-24 18:01:58.027897+00:00   \n",
      "2       1003 2024-09-24 18:01:58.027897+00:00   \n",
      "\n",
      "   label_driver_reported_satisfaction  val_to_add  val_to_add_2  conv_rate  \\\n",
      "0                                   5           2            20   0.311688   \n",
      "1                                   1           1            10   0.776248   \n",
      "2                                   3           3            30   0.235401   \n",
      "\n",
      "   acc_rate  avg_daily_trips  conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0  0.991556              579             2.311688            20.311688  \n",
      "1  0.120384              311             1.776248            10.776248  \n",
      "2  0.644993              381             3.235401            30.235401  \n"
     ]
    }
   ],
   "source": [
    "entity_df[\"event_timestamp\"] = pd.to_datetime(\"now\", utc=True)\n",
    "training_df = store.get_historical_features(\n",
    "    entity_df=entity_df,\n",
    "    features=[\n",
    "        \"driver_hourly_stats:conv_rate\",\n",
    "        \"driver_hourly_stats:acc_rate\",\n",
    "        \"driver_hourly_stats:avg_daily_trips\",\n",
    "        \"transformed_conv_rate:conv_rate_plus_val1\",\n",
    "        \"transformed_conv_rate:conv_rate_plus_val2\",\n",
    "    ],\n",
    ").to_df()\n",
    "\n",
    "print(\"\\n----- Example features -----\\n\")\n",
    "print(training_df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56bd764f-4455-4ade-b630-81fdbb2eb444",
   "metadata": {},
   "source": [
    "## Ingest batch features into your online store"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e5595e-589e-4639-b631-073183f8240d",
   "metadata": {},
   "source": [
    "This command will generate the features from offline store and stores into online store. This command will call `get_historical_features` to get the data from offline store."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b95b0b1b-d608-429d-91ac-ef96c698d9ca",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "09/24/2024 06:02:09 PM root WARNING: _list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n",
      "Materializing \u001b[1m\u001b[32m2\u001b[0m feature views to \u001b[1m\u001b[32m2024-09-24 18:02:06+00:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
      "\n",
      "\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m from \u001b[1m\u001b[32m2024-09-23 18:02:09+00:00\u001b[0m to \u001b[1m\u001b[32m2024-09-24 18:02:06+00:00\u001b[0m:\n",
      "  0%|                                                                         | 0/5 [00:00<?, ?it/s]09/24/2024 06:02:09 PM root WARNING: Cannot use sqlite_vec for vector search\n",
      "100%|████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 670.38it/s]\n",
      "\u001b[1m\u001b[32mdriver_hourly_stats_fresh\u001b[0m from \u001b[1m\u001b[32m2024-09-23 18:02:09+00:00\u001b[0m to \u001b[1m\u001b[32m2024-09-24 18:02:06+00:00\u001b[0m:\n",
      "100%|████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 807.56it/s]\n"
     ]
    }
   ],
   "source": [
    "!feast materialize-incremental $(date -u +\"%Y-%m-%dT%H:%M:%S\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59890c1d-fb96-4a56-a11a-ffb0883c3b04",
   "metadata": {},
   "source": [
    "## Fetching feature vectors for inference\n",
    "At inference time, we need to quickly read the latest feature values for different drivers (which otherwise might have existed only in batch sources) from the online feature store using `get_online_features()`. These feature vectors can then be fed to the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "18a57a9c-4637-4fe3-a37d-e2f51c8e8dd6",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n",
      "WARNING:root:Cannot use sqlite_vec for vector search\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'acc_rate': [0.49898454546928406, 0.2943153381347656],\n",
      " 'avg_daily_trips': [178, 74],\n",
      " 'conv_rate': [0.19129787385463715, 0.5790505409240723],\n",
      " 'driver_id': [1004, 1005]}\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "from feast import FeatureStore\n",
    "\n",
    "store = FeatureStore(repo_path=\".\")\n",
    "\n",
    "feature_vector = store.get_online_features(\n",
    "    features=[\n",
    "        \"driver_hourly_stats:conv_rate\",\n",
    "        \"driver_hourly_stats:acc_rate\",\n",
    "        \"driver_hourly_stats:avg_daily_trips\",\n",
    "    ],\n",
    "    entity_rows=[\n",
    "        # {join_key: entity_value}\n",
    "        {\"driver_id\": 1004},\n",
    "        {\"driver_id\": 1005},\n",
    "    ],\n",
    ").to_dict()\n",
    "\n",
    "pprint(feature_vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "239ed52a-1460-4233-880e-620135b540b2",
   "metadata": {},
   "source": [
    "## Using a feature service to fetch online features instead.\n",
    "You can also use feature services to manage multiple features, and decouple feature view definitions and the features needed by end applications. The feature store can also be used to fetch either online or historical features using the same API below. \n",
    "\n",
    "The `driver_activity_v4` feature service pulls all features from the `driver_hourly_stats` feature view:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "3878578f-bba6-4f48-a458-a2b69fc90f0a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Cannot use sqlite_vec for vector search\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FeatureService driver_activity_v4 created.\n"
     ]
    }
   ],
   "source": [
    "import example_repo\n",
    "from feast import FeatureStore\n",
    "\n",
    "\n",
    "from feast import FeatureService\n",
    "driver_activity_v4 = FeatureService(\n",
    "    name=\"driver_activity_v4\",\n",
    "    features=[example_repo.driver_stats_fresh_fv],\n",
    ")\n",
    "\n",
    "feature_store = FeatureStore('.')  # Initialize the feature store\n",
    "\n",
    "feature_store.apply([driver_activity_v4])\n",
    "\n",
    "print(\"FeatureService driver_activity_v4 created.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "974a3d09-2237-460a-a310-bdb1a13c6b85",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'acc_rate': [0.49898454546928406, 0.2943153381347656],\n",
      " 'avg_daily_trips': [178, 74],\n",
      " 'conv_rate': [0.19129787385463715, 0.5790505409240723],\n",
      " 'driver_id': [1004, 1005]}\n"
     ]
    }
   ],
   "source": [
    "import example_repo\n",
    "from pprint import pprint\n",
    "\n",
    "\n",
    "#feature_service = feature_store.get_feature_service(\"driver_activity_v4\")\n",
    "feature_vector = feature_store.get_online_features(\n",
    "    features=driver_activity_v4,\n",
    "    entity_rows=[\n",
    "        # {join_key: entity_value}\n",
    "        {\"driver_id\": 1004},\n",
    "        {\"driver_id\": 1005},\n",
    "    ],\n",
    ").to_dict()\n",
    "pprint(feature_vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf289369-bb77-4dfa-9ae1-994ddc9fa79a",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Accessing Features using remote online store\n",
    "\n",
    "In this section we will start the feast in server and client mode. We will start the feast online server and retrieve online features using `remote` online store.\n",
    "\n",
    "By default online server starts on the port: `6566`. We are going to still refer the same registry as `my_feast_project` to keep this example simple to understand instead of starting registry and online server at the same time. You can review the client feature store configuration [here](./remote-online/feature_store.yaml). \n",
    "\n",
    "\n",
    "In the actual production environment you can run registry, online and offline servers and access them remotely using feature store clients. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f6dd4fd-b018-43c2-8467-b2f4d3a0994d",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Starting feast online feature server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "a3243672-4877-4588-b628-4d344bb55c7c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/app-root/lib64/python3.9/site-packages/uvicorn/workers.py:16: DeprecationWarning: The `uvicorn.workers` module is deprecated. Please use `uvicorn-worker` package instead.\n",
      "For more details, see https://github.com/Kludex/uvicorn-worker.\n",
      "  warnings.warn(\n",
      "[2024-09-24 18:03:30 +0000] [17522] [INFO] Starting gunicorn 23.0.0\n",
      "[2024-09-24 18:03:30 +0000] [17522] [INFO] Listening at: http://127.0.0.1:6566 (17522)\n",
      "[2024-09-24 18:03:30 +0000] [17522] [INFO] Using worker: uvicorn.workers.UvicornWorker\n",
      "[2024-09-24 18:03:30 +0000] [17570] [INFO] Booting worker with pid: 17570\n",
      "[2024-09-24 18:03:30 +0000] [17570] [INFO] Started server process [17570]\n",
      "[2024-09-24 18:03:30 +0000] [17570] [INFO] Waiting for application startup.\n",
      "[2024-09-24 18:03:30 +0000] [17570] [INFO] Application startup complete.\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "\n",
    "# Run feast serve in the background\n",
    "feast_online_server_process = subprocess.Popen([\"feast\", \"serve\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "cde4756b-ac6a-490b-bbcc-89d9b1ba3285",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1001130+   17522   16173 32 18:03 ?        00:00:04 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve\n",
      "1001130+   17570   17522  0 18:03 ?        00:00:00 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve\n",
      "1001130+   17578   17575  0 18:03 ?        00:00:00 grep feast serve\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "# checking if the online server process started.\n",
    "ps -ef | grep 'feast serve'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee01d7e9-90cf-405e-8f0b-bcf648f45d52",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Retrieving the features using online remote client "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "fbd81a9e-a388-4a1b-a2a0-3f1f838194d7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "remote-online feature_Store.yaml file has been created.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import yaml\n",
    "\n",
    "directory = os.path.abspath(\"./../../remote-online\")\n",
    "os.makedirs(directory, exist_ok=True)\n",
    "\n",
    "data = {\n",
    "    'project': 'my_feast_project',\n",
    "    'registry': './../my_feast_project/feature_repo/data/registry.db',\n",
    "    'provider': 'local',\n",
    "    'online_store': {\n",
    "        'type': 'remote',\n",
    "        'path': 'http://127.0.0.1:6566'\n",
    "    },\n",
    "    'entity_key_serialization_version': 3\n",
    "}\n",
    "\n",
    "file_path = os.path.join(directory, 'feature_store.yaml')\n",
    "\n",
    "# Write to a YAML file\n",
    "with open(file_path, 'w') as file:\n",
    "    yaml.dump(data, file, default_flow_style=False)\n",
    "\n",
    "print(\"remote-online feature_Store.yaml file has been created.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "2a30b4d8-78f4-4bff-afb5-681cf951a812",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/remote-online\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/app-root/lib64/python3.9/site-packages/IPython/core/magics/osm.py:417: UserWarning: using dhist requires you to install the `pickleshare` library.\n",
      "  self.shell.db['dhist'] = compress_dhist(dhist)[-100:]\n"
     ]
    }
   ],
   "source": [
    "%cd ./../../remote-online"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "e97fc9c6-2b3c-4acd-a095-fcbee2d9b2be",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "remote online feature store client has been initialized.\n"
     ]
    }
   ],
   "source": [
    "online_feature_store_client = FeatureStore('.')\n",
    "online_feature_store_client.apply([])\n",
    "print(\"remote online feature store client has been initialized.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac28dac4-0d6d-4340-a883-60444887a01a",
   "metadata": {},
   "source": [
    "Now we are going to retrieve the same features we have retrieved in previous section. Here we are client store going to retrieve the features using remote feature store."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "698f5462-ad4a-40cb-955e-10353b765764",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127.0.0.1:55838 - \"POST /get-online-features HTTP/1.1\" 200\n",
      "{'acc_rate': [0.49898454546928406, 0.2943153381347656],\n",
      " 'avg_daily_trips': [178, 74],\n",
      " 'conv_rate': [0.19129787385463715, 0.5790505409240723],\n",
      " 'driver_id': [1004, 1005]}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "09/24/2024 06:04:33 PM root WARNING: _list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n",
      "09/24/2024 06:04:33 PM root WARNING: Cannot use sqlite_vec for vector search\n"
     ]
    }
   ],
   "source": [
    "online_features_stores_client = online_feature_store_client.get_online_features(\n",
    "    features=driver_activity_v4,\n",
    "    entity_rows=[\n",
    "        # {join_key: entity_value}\n",
    "        {\"driver_id\": 1004},\n",
    "        {\"driver_id\": 1005},\n",
    "    ],\n",
    ").to_dict()\n",
    "pprint(online_features_stores_client)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bf1ad00-cd37-4feb-bd5d-4b00854e34dd",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Accessing Feast Registry metadata using remote registry store\n",
    "Registry is going to have all the metadata information of feast objects such as FeatureService, FeatureViews. Either you can directly access this information using the way referred in above section.\n",
    "\n",
    "The other way to access in the client server model. You can start the registry server and access them using remote registry client as shown in this section.\n",
    "\n",
    "The default port for the registry is `6570`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2e068e2-e3cd-4444-81b3-f335d27d47a8",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Starting the registry server as remote"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a401abd1-c250-46a9-9ed1-3f4c3b705cec",
   "metadata": {},
   "source": [
    "Change the current directory context to initial feature store so that we can start the registry server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "be7cf013-33db-487a-97cb-48d3b23924e1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo\n"
     ]
    }
   ],
   "source": [
    "%cd ./../my_feast_project/feature_repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "835c26e8-1df9-4b02-967a-08eb9747c258",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Registry server started on the default port 6570. Go to next cell and check if the process is available.\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "\n",
    "# Run feast serve in the background\n",
    "feast_remote_registry_server_process = subprocess.Popen([\"feast\", \"serve_registry\"])\n",
    "print(\"Registry server started on the default port 6570. Go to next cell and check if the process is available.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "6330dea7-375d-4d50-888b-7f1d126a3a83",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo\n",
      "1001130+   17902   16173  1 18:07 ?        00:00:04 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve_registry\n",
      "1001130+   18249   18246  0 18:13 ?        00:00:00 grep feast serve_registry\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "# checking if the registry server process started.\n",
    "pwd\n",
    "ps -ef | grep 'feast serve_registry'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3d90ca8-c8ff-4c37-b72f-64e06cf8f97a",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Initializing the remote registry client and retrieving the feast metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "392bf771-ecfb-43f3-8f87-b6762b12f5f3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "remote-registry feature_Store.yaml file has been created.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import yaml\n",
    "\n",
    "directory = os.path.abspath(\"./../../remote-registry\")\n",
    "os.makedirs(directory, exist_ok=True)\n",
    "\n",
    "data = {\n",
    "    'project': 'my_feast_project',\n",
    "    'registry': {\n",
    "        'registry_type': 'remote',\n",
    "        'path': 'localhost:6570'\n",
    "    },\n",
    "    'provider': 'local',\n",
    "    'online_store': {\n",
    "        'type': 'remote',\n",
    "        'path': 'http://127.0.0.1:6566'\n",
    "    },\n",
    "    'entity_key_serialization_version': 3\n",
    "}\n",
    "\n",
    "file_path = os.path.join(directory, 'feature_store.yaml')\n",
    "\n",
    "# Write to a YAML file\n",
    "with open(file_path, 'w') as file:\n",
    "    yaml.dump(data, file, default_flow_style=False)\n",
    "\n",
    "print(\"remote-registry feature_Store.yaml file has been created.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "4cce650d-bcbc-4ea7-8a5e-168ebb561765",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/remote-registry\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/app-root/lib64/python3.9/site-packages/IPython/core/magics/osm.py:417: UserWarning: using dhist requires you to install the `pickleshare` library.\n",
      "  self.shell.db['dhist'] = compress_dhist(dhist)[-100:]\n"
     ]
    }
   ],
   "source": [
    "%cd ./../../remote-registry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "39e255c9-7172-4ec2-9fee-5f30f2f465b3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Remote registry feature store client has been initialized.\n"
     ]
    }
   ],
   "source": [
    "registry_feature_store_client = FeatureStore('.')\n",
    "registry_feature_store_client.apply([])\n",
    "print(\"Remote registry feature store client has been initialized.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "e6778cb5-cee1-4010-bf6b-f81ef3fa38bd",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:_list_feature_views will make breaking changes. Please use _list_batch_feature_views instead. _list_feature_views will behave like _list_all_feature_views in the future.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<FeatureView(name = driver_hourly_stats, entities = ['driver'], ttl = 1 day, 0:00:00, stream_source = None, batch_source = {\n",
       "   \"type\": \"BATCH_FILE\",\n",
       "   \"timestampField\": \"event_timestamp\",\n",
       "   \"createdTimestampColumn\": \"created\",\n",
       "   \"fileOptions\": {\n",
       "     \"uri\": \"/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo/data/driver_stats.parquet\"\n",
       "   },\n",
       "   \"name\": \"driver_hourly_stats_source\"\n",
       " }, entity_columns = [driver_id-Int64], features = [conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], description = , tags = {'team': 'driver_performance'}, owner = , projection = FeatureViewProjection(name='driver_hourly_stats', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={}), created_timestamp = 2024-09-24 18:01:41.750517, last_updated_timestamp = 2024-09-24 18:02:09.176556, online = True, materialization_intervals = [(datetime.datetime(2024, 9, 23, 18, 2, 9, 112629, tzinfo=<UTC>), datetime.datetime(2024, 9, 24, 18, 2, 6, tzinfo=<UTC>))])>,\n",
       " <FeatureView(name = driver_hourly_stats_fresh, entities = ['driver'], ttl = 1 day, 0:00:00, stream_source = {\n",
       "   \"type\": \"PUSH_SOURCE\",\n",
       "   \"name\": \"driver_stats_push_source\",\n",
       "   \"batchSource\": {\n",
       "     \"type\": \"BATCH_FILE\",\n",
       "     \"timestampField\": \"event_timestamp\",\n",
       "     \"createdTimestampColumn\": \"created\",\n",
       "     \"fileOptions\": {\n",
       "       \"uri\": \"/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo/data/driver_stats.parquet\"\n",
       "     },\n",
       "     \"name\": \"driver_hourly_stats_source\"\n",
       "   }\n",
       " }, batch_source = {\n",
       "   \"type\": \"BATCH_FILE\",\n",
       "   \"timestampField\": \"event_timestamp\",\n",
       "   \"createdTimestampColumn\": \"created\",\n",
       "   \"fileOptions\": {\n",
       "     \"uri\": \"/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo/data/driver_stats.parquet\"\n",
       "   },\n",
       "   \"name\": \"driver_hourly_stats_source\"\n",
       " }, entity_columns = [driver_id-Int64], features = [conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], description = , tags = {'team': 'driver_performance'}, owner = , projection = FeatureViewProjection(name='driver_hourly_stats_fresh', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={}), created_timestamp = 2024-09-24 18:01:41.750653, last_updated_timestamp = 2024-09-24 18:02:09.228691, online = True, materialization_intervals = [(datetime.datetime(2024, 9, 23, 18, 2, 9, 176964, tzinfo=<UTC>), datetime.datetime(2024, 9, 24, 18, 2, 6, tzinfo=<UTC>))])>,\n",
       " <OnDemandFeatureView(name = transformed_conv_rate, features = [conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], description = , tags = {}, owner = , projection = FeatureViewProjection(name='transformed_conv_rate', name_alias=None, desired_features=[], features=[conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], join_key_map={}), created_timestamp = 2024-09-24 18:01:41.750805, last_updated_timestamp = 2024-09-24 18:01:41.750805, mode = pandas, source_feature_view_projections = {'driver_hourly_stats': FeatureViewProjection(name='driver_hourly_stats', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={})}, source_request_sources = {'vals_to_add': <feast.data_source.RequestSource object at 0x7f7ccc7dda00>}, feature_transformation = <feast.transformation.pandas_transformation.PandasTransformation object at 0x7f7ccc7dd910>)>,\n",
       " <OnDemandFeatureView(name = transformed_conv_rate_fresh, features = [conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], description = , tags = {}, owner = , projection = FeatureViewProjection(name='transformed_conv_rate_fresh', name_alias=None, desired_features=[], features=[conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], join_key_map={}), created_timestamp = 2024-09-24 18:01:41.751687, last_updated_timestamp = 2024-09-24 18:01:41.751687, mode = pandas, source_feature_view_projections = {'driver_hourly_stats_fresh': FeatureViewProjection(name='driver_hourly_stats_fresh', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={})}, source_request_sources = {'vals_to_add': <feast.data_source.RequestSource object at 0x7f7ccc7ddeb0>}, feature_transformation = <feast.transformation.pandas_transformation.PandasTransformation object at 0x7f7ccc7dd9d0>)>]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Listing all feature views using remote registry client\n",
    "registry_feature_store_client.list_all_feature_views(allow_cache=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "31889388-5efc-423c-b64e-e8e5c0b67a68",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<FeatureService(name = driver_activity_v2, _features = [], feature_view_projections = [FeatureViewProjection(name='driver_hourly_stats', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={}), FeatureViewProjection(name='transformed_conv_rate', name_alias=None, desired_features=[], features=[conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], join_key_map={})], description = , tags = {}, owner = , created_timestamp = 2024-09-24 18:01:41.752541, last_updated_timestamp = 2024-09-24 18:01:41.752541, logging_config = None)>,\n",
       " <FeatureService(name = driver_activity_v1, _features = [], feature_view_projections = [FeatureViewProjection(name='driver_hourly_stats', name_alias=None, desired_features=[], features=[conv_rate-Float32], join_key_map={}), FeatureViewProjection(name='transformed_conv_rate', name_alias=None, desired_features=[], features=[conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], join_key_map={})], description = , tags = {}, owner = , created_timestamp = 2024-09-24 18:01:41.752667, last_updated_timestamp = 2024-09-24 18:01:41.752667, logging_config = <feast.feature_logging.LoggingConfig object at 0x7f7ccc7e2580>)>,\n",
       " <FeatureService(name = driver_activity_v3, _features = [], feature_view_projections = [FeatureViewProjection(name='driver_hourly_stats_fresh', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={}), FeatureViewProjection(name='transformed_conv_rate_fresh', name_alias=None, desired_features=[], features=[conv_rate_plus_val1-Float64, conv_rate_plus_val2-Float64], join_key_map={})], description = , tags = {}, owner = , created_timestamp = 2024-09-24 18:01:41.752760, last_updated_timestamp = 2024-09-24 18:01:41.752760, logging_config = None)>,\n",
       " <FeatureService(name = driver_activity_v4, _features = [], feature_view_projections = [FeatureViewProjection(name='driver_hourly_stats_fresh', name_alias=None, desired_features=[], features=[conv_rate-Float32, acc_rate-Float32, avg_daily_trips-Int64], join_key_map={})], description = , tags = {}, owner = , created_timestamp = 2024-09-24 18:02:28.776861, last_updated_timestamp = 2024-09-24 18:02:28.776861, logging_config = None)>]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Listing all feature services using remote registry client\n",
    "registry_feature_store_client.list_feature_services()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a074f5bc-1d1b-4ad1-9b08-b3aea58761e8",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Stopping the online, registry server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "d8f353e0-3874-49f7-af82-bd79396eb6d8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo\n",
      "1001130+   17522   16173  0 18:03 ?        00:00:04 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve\n",
      "1001130+   17570   17522  0 18:03 ?        00:00:00 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve\n",
      "1001130+   17902   16173  0 18:07 ?        00:00:04 /opt/app-root/bin/python3.9 /opt/app-root/bin/feast serve_registry\n",
      "1001130+   18481   18438  0 18:15 ?        00:00:00 grep feast serve\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "# checking if the registry server and online server process is already running.\n",
    "pwd\n",
    "ps -ef | grep 'feast serve'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "8c84142d-b835-440e-9177-9b8bfe00768f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "remote online and registry server has been stopped.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2024-09-24 18:16:00 +0000] [17522] [INFO] Handling signal: term\n",
      "[2024-09-24 18:16:00 +0000] [17570] [INFO] Shutting down\n",
      "[2024-09-24 18:16:00 +0000] [17570] [INFO] Error while closing socket [Errno 9] Bad file descriptor\n",
      "[2024-09-24 18:16:00 +0000] [17570] [INFO] Waiting for application shutdown.\n",
      "[2024-09-24 18:16:00 +0000] [17570] [INFO] Application shutdown complete.\n",
      "[2024-09-24 18:16:00 +0000] [17570] [INFO] Finished server process [17570]\n",
      "[2024-09-24 18:16:00 +0000] [17522] [ERROR] Worker (pid:17570) was sent SIGTERM!\n",
      "[2024-09-24 18:16:00 +0000] [17522] [INFO] Shutting down: Master\n"
     ]
    }
   ],
   "source": [
    "feast_online_server_process.terminate()  # Stop the remote Feast online server\n",
    "feast_remote_registry_server_process.terminate() # stops the remote registry server\n",
    "print(\"remote online and registry server has been stopped.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "93bf01b6-5bf1-4345-9bef-2e83edc489b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/app-root/src/feast/examples/rhoai-quickstart/my_feast_project/feature_repo\n",
      "1001130+   18542   18499  0 18:16 ?        00:00:00 grep feast serve\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "# checking if the registry server and online server process stopped. wait for some time until it kills.\n",
    "pwd\n",
    "ps -ef | grep 'feast serve'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbe06b35-7586-4b26-a63b-29bcd7ea3037",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
